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Abstract 

Elliptically contoured distributions can be considered to be the distributions for 
which the contours of the density functions are proportional elhpsoids. Kamiya, 
Takemura and Kuriki (2006) generahzed the elhptically contoured distributions to 
star-shaped distributions, for which the contours are allowed to be arbitrary pro- 
portional star-shaped sets. This was achieved by considering the so-called orbital 
decomposition of the sample space in the general framework of group invariance. 
In the present paper, we extend their results by conducting the orbital decompo- 
sitions in steps and obtaining a further, hierarchical decomposition of the sample 
space. This allows us to construct probability models and distributions with fur- 
ther independence structures. The general results are applied to the star-shaped 
distributions with a certain symmetric structure, the distributions related to the 
two-sample Wishart problem and the distributions of preference rankings. 
Key words: action, decomposable distribution, elliptically contoured distribution, 
global cross section, Haar measure, isotropy subgroup, orbital decomposition, rank- 
ing, star-shaped distribution. 



1 Introduction 

Elliptically contoured distributions are defined to be the distributions for which the con- 
tours of the density functions are proportional ellipsoids. As a natural generalization of 
the multivariate normal distributions, they are widely used as a distributional assumption 
(Kelker (1970), Cambanis, Huang and Simons (1981), Fang and Anderson (1990)). Fang 
and Zhang (1990) discuss "generalized multivariate analysis" based on elliptically con- 
toured distribuitons. In the meantime, from another perspective, elliptically contoured 
distributions can be obtained from spherical distributions by affine transformation. Ex- 
tending the /2-iiorm in spherical distributions to the /g-norm, g > 0, Osiewalski and 
Steel (1993) introduced /^-spherical distributions. 



1 



Generalizing elliptically contoured distributions and /^-spherical distributions, Kamiya, 
Takemura and Kuriki (2006) defined the so-called star-shaped distributions, for which the 
contours of the density functions arc proportional to (the boundaries of) arbitrary star- 
shaped sets called cross sections (see also Fernandez, Osiewalski and Steel (1995) and 
Ferreira and Steel (2005)). They showed that basic facts about the independence of the 
"length" and "direction" continue to hold for star-shaped distributions. However, when 
the star-shaped set has a symmetric structure, we can make a more detailed investigation 
into this distribution. 

In the star-shaped distribution, the cross section C — {0} is allowed to be 
an arbitrary star-shaped set — a set which intersects each ray emanating from the origin 
exactly once, and the density is assumed to be constant on each proportional star-shaped 
set gZ — {gz : z e Z}, g > 0. However, there are some cases where we have some 
symmetry; as in the case of elliptically contoured distributions, we might be able to 
assume that Z is symmetric about the origin: Z = —Z = {—z : z G Z}. In those 
cases, Z may be obtained as Z = {±1}Z = {±z : z G Z} = Z U {—Z) in terms of a 
set Z C MP — {0} which intersects each hue through the origin exactly once. As long 
as this condition is satisfied, Z is allowed to be an arbitrary set. Now, suppose x eW 
is distributed according to a star-shaped distribution with respect to such a symmetric 
Z = Z U {—Z). Then the density of this distribution is constant on each gZ, g > 0, 
and the distribution of —x is the same as that of x. 

In the above situation, we cannot deal with the skewness of the distributions. However, 
we can go further and consider those distributions whose densities are constant on each 
gZ, g 0, but not necessarily constant on each gZ — gZ U {—gZ), g > 0, where 
Z = Z U {—Z). That is, the value of the density on gZ can differ from the value on 
—gZ for g > 0. In such a case, distributions of x and —x are not the same. 

These types of distributions can be studied by decomposing x (7^ 0) uniquely as 
X — ehz, e — ±1, /i > 0, z & Z, with respect to Z. As with Kamiya, Takemura and 
Kuriki (2006), these problems can be treated as special cases of a general discussion in 
terms of abstract group invariance, and we choose to do so in this paper. This approach 
enables us to apply the obtained general results to the distributions of random matrices 
and moreover discrete distributions. 

In the general framework of group invariance, the present problem corresponds to 
the decomposition of the sample space under the action of an invariance group, called the 
orbital decomposition (Wijsman (1990)). Conducting this decomposition twice, we obtain 
a hierarchical decomposition into three parts. When a group Q acts on a sample space 
A", any subgroup Ti oi Q acts on each ^-orbit. By choosing an appropriate and 
performing a hierarchical orbital decomposition, we can construct probability models with 
the corresponding hierarchical independence structures. This allows us to propose in our 
general framework new probability models for various statistical problems, as illustrated 
with modeling of preference rankings in Section 4.3. 

The organization of this paper is as follows. In Section 2, we summarize some funda- 
mental facts about group actions and orbital decompositions, and review the results about 
the decomposable distributions studied in Kamiya, Takemura and Kuriki (2006). In Sec- 
tion 3 we introduce a further, hierarchical decomposition by means of a subgroup action 
and define extended decomposable distributions. We establish various facts on hierarchi- 
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cal orbital decompositions and derive distributional properties of extended decomposable 
distributions. In the final section, we apply the general results to the star-shaped distri- 
butions (Section 4.1), the two-sample Wishart problem (Section 4.2) and the distributions 
of preference rankings (Section 4.3). 

2 Orbital decomposition and decomposable distribu- 
tion 

In this section we summarize some fundamental facts about group actions and orbital 
decompositions, and review the results about the decomposable distributions. For group 
invariance in statistics, the reader is referred to Eaton (1989), Barndorff-Nielsen, Blaesild 
and Eriksen (1989) and Wijsman (1990). For global cross sections and orbital decomposi- 
tions in particular, see Wijsman (1967, 1986), Koehn (1970), Bondar (1976) and Kamiya 
(1996). 

2.1 Orbital decomposition 

Let a group Q act on a space X (typically the sample space) from the left {g,x) ^-^ gx : 
Q X X ^ X. We write the action of ^ on A' as 

Let Qx = {gx : g E Q} be the or&i^ containing x G X, and let X/Q = {Qx : x e X} 
be the orhit space, i.e., the set of all orbits. When X consists of a single orbit X — Qx, 
the action is said to be transitive. 

Indicate by Qx = {q ^ Q '■ = x} the isotropy subgroup at x E X. When Q^ = {e} 
for all X E X, the action is said to be free, where e denotes the identity element of Q. 
In general, the isotropy subgroups at two points on a common orbit are conjugate to each 
other: Qg^ = gQxg~^, 9 eQ, x e X. 

The set of left cosets gQ^ = {gg' '■ g' E Qx}, g E Q, is called the left coset space of Q 
modulo Qx, and is denoted by Q/Qx = {gQx '■ 9 E Q}. The canonical map tt : Q ^ Q /Qx 
is defined by n^g) = gQx, g E Q. The group Q or more generally its subgroup 7i < Q 
acts on Q/Qx by 



This action is not transitive unless Ti, includes a complete set of representatives of 

gQx, 9^Q, i-e., G = [jh&H^'3x- 

A subset Z <Z X IS said to be a cross section if Z intersects each orbit Qx, x E X, 
exactly once. So any cross section Z is in one-to-one correspondence with the orbit space 
X/Q by z ^ Qz, z E Z. A cross section Z having a common isotropy subgroup is 
called a global cross section: Qz = Qq, say, for all z E Z. Unlike a mere cross section, a 
global cross section does not always exist. A global cross section exists if and only if the 
isotropy subgroups Qx, x E X, are all conjugate to one another. 

Unless otherwise stated, however, we assume from now on that there does exist a 
global cross section Z. Then, we have the following one-to-one correspondence: 



(1) 



{h,gQx) ^ {hg)Qx, hEH, g eQ. 



(2) 



X ^ yxz. 



X <-> 



{y,z), x^gz, y^n{g), gEQ, 
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where 3^ = Q /Qi) with Qq = Qz, z E Z. Decomposition Q is called the orbital decom- 
position of X (or x) with respect to Z. In (j2I) we can think of y and z as functions 
y = y{x) and z = z{x) of x. Under the action of Q on X, y{x) is equivariant and 
z{x) is invariant: y{gx) = gy{x), z{gx) = z{x), g E G, x E X. 

We move on to reviewing some properties of global cross sections obtained in Kamiya, 
Takemura and Kuriki (2006). 

Let be a global cross section. Then gZ = {gz : z G Z} for each g E Q is 
again a global cross section. We say gZ, g E Q, are proportional to Z, and call 
{gZ '■ g E Q} the family of proportional global cross sections. In A" = [J^^ggZ, it holds 
that giZ n g2Z ^ for gi,g2 G Q implies giZ = g2Z, so the family of proportional 
global cross sections gives a partition of X. 

From a given global cross section Z, we can construct a general cross section Z' by 
changing the points of Z within their orbits. For Z' to be global, i.e., for the isotropy 
subgroups to be the same on the whole of Z', these changes of the points have to be 
made subject to some restriction as follows. Let Af = {g E Q : gQog~^ = Qo} be the 
normalizer of the common isotropy subgroup Qo on Z. Then a subset Z' G X is a 
global cross section if and only if it can be written as 



for some go E Q and E Af, z E Z. 

Under the change from Z to Z' in Q, the equivariant part transforms as follows. 
Let X ^ {y, z) be the orbital decomposition with respect to Z, and let x ^ {y', z') be 
the orbital decomposition with respect to the Z' in Then we have 



2.2 Decomposable distribution 

In this subsection, we review the decomposable distributions defined in Kamiya, Takemura 
and Kuriki (2006). 

Throughout the rest of the paper, we make the following assumptions: (a) X is a. 
locally compact Hausdorff space; (b) ^ is a second countable, locally compact Hausdorff 
topological group acting continuously on X; (c) Qq is compact; and (d) Z is locally 
compact and the bijection x [y, z) with respect to Z is bimeasurable. 

We consider distributions on X which have densities f{x) with respect to a domi- 
nating measure A. Measure A is assumed to be relatively invariant with multiplier x '■ 
\{d{gx)) = x{g)^{dx), g E Q. Then, we say a distribution f{x)X{dx) is a decomposable 
distribution with respect to Z if it is of the form f{x)X{dx) = fy{y{x))fz{z{x))X{dx). 
In particular, we say it is cross-sectionally contoured if fz{z) is constant, and orbitally 
contoured if fy{y) is constant. We mainly study cross-sectionally contoured distribu- 
tions because a decomposable distribution fy{y{x))fz{z{x))X{dx) can always be thought 
of as a cross-sectionally contoured distribution with density fy{y{x)) with respect to 
X{dx) := fz{z{x))X{dx) Obviously, a distribution f{x)X{dx) is cross-sectionally con- 
toured with respect to Z if and only if /(x) is constant on each proportional global 
cross section gZ, g E Q. 



(3) 



^' = {go^zZ : z E Z} 



(4) 



y' = y^z ^9o ^ 
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Topological assumption (b) about Q implies that there exists a left Haar measure 
fig on Q, which is unique up to a multiplicative constant. By the compactness of Qq 
assumed in (c), we have the induced measure fiy = 7r(/ig) = fign'^ on 3^ (Proposition 
2.3.5 and Corollary 7.4.4 of Wijsman (1990)). Again by the same assumption (c), we can 
define xiv): 2/^3^; by xiu) = xi.9) with g G Ti^^{{y}). By abuse of notation, we will 
write xiy) for xiy)- 

In terms of these, \{dx) is factored as 

(5) \{dx) = xiy)l^yidy)uzidz) 

(Theorem 7.5.1 of Wijsman (1990), Theorem 10.1.2 of Farrell (1985)). Here, we are 
identifying X with y x Z. Existence of a density f{x) with respect to A implies 
uz is a finite measure, so from now on we assume that vz{dz) is standardized to be a 
probability measure on Z. From (jSJ we immediately obtain the following result. 

Proposition 2.1. (Kamiya, Takemura and Kuriki (2006)) Suppose that x is 
distributed according to a cross-sectionally contoured distribution fy{y{x))X{dx). Then 
we have: 

1. y = y{x) and z = z{x) are independently distributed. 

2. The distribution of y is fy{y)x{.y)l^y{.dy). 

3. The distribution of z does not depend on fy. 

Note that since uzidz) is taken to be a probability measure, we do not need a 
normalizing constant in fy{y)x{y)l^y{dy)- (To put it another way, the version of fiy 
is taken in this way.) 

3 Hierarchical orbital decomposition and extended 
decomposable distribution 

In this section we introduce a further, hierarchical decomposition and define extended 
decomposable distributions. 



3.1 Hierarchical orbital decomposition 

In this subsection, we give a further factorization of the ^-orbital decomposition. This is 
obtained by decomposing the equivariant part Q/Qo by means of the action of a subgroup 

of g. 

We continue to assume that there exists a global cross section Z with the common 
isotropy subgroup Qq. Furthermore, let H he a. subgroup of Q. 
As in Section 2.1, we have the decomposition 

(6) X ^ g/Go X z. 

Now, Ti. acts on Q/Qo by with = Go- Note that instead of this action we may 
equivalently consider the action of Ti on Qzq : {h,gzo) {hg)zo, Zq G Z. In particular, 
we have Ti^go = Ti-gzo, 9 ^ Si from which we obtain the following lemma: 
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Lemma 3.1. Action (7i, Af) is free if and only if action {T-l,Q/Qo) is free. 

Now suppose a global cross section V C G/Go exists for action {Ti.,Q/Qo). The 
existence of a global cross section V leads to a further decomposition of (jH)) as follows. 

Denote the common isotropy subgroup at the points of V by Hq. Then Q/Qo is 
decomposed as 

(7) g/Go ^ n/Ho X V. 

We can take V in such a way that Qq G V; in that case, we can write Tio as 

Ho = Kg, = {hen-. hGo = go} = nr] Go. 

From now on, we always take V in this way. 

Combining ^ and ((Zj), we have the decomposition 

(8) X ^ n/Ho xVxZ. 
Our questions are: 

(i) specifying the condition for V to exist, and 

(ii) expressing V in a concrete form. 

Note that the orbits under (H, G/Go) are of the form 

{hgGo ■■hen}c G/Go, g^G. 

This suggests that the above questions are closely related to the properties of the double 
cosets 

T-CgGo = {hggo : heH, go e Go} 

= n-^{{hgGo-hen}) cG, gEG, 

in G- The following lemma indicates this fact. 

Lemma 3.2. Let G' C G- Then V = {g'Go '■ g' ^ G'} C G/Go is a cross section for the 
action of H on G/Go such that g'Go 7^ g"Go for g' ^ g" , g' ,g" G G' ■, if and only if G' 
is a complete set of representatives of the double cosets TigGoi 9 ^ G, in G '■ 

G = \ I 'Hg'Go (disjoint union). 

g'eg' 

Proof. Necessity: Suppose V = {g'Go '■ g' £ G'} is a cross section for action (7i, G/Go) 
and that g'Go = g"Go for g', g" G G' implies g' = g". We want to prove G = 
Ug'eg' Hg'Go- It suffices to verify (a) G C Ug'eg' H.g'Go', and (b) Hg'Go = H.g"Go for 
g'^g" £ G' implies g' = g". First, (a) is shown as follows. Let g be an arbitrary el- 
ement of G- Then, since V intersects the orbit containing gGo ^ G/Go at least once, 
there exist h E H and g' G G' such that gGo = hg'Go- Thus g can be written as 
g = kg' go with some go G Go- Therefore, g G Hg'Go C \Jgii(,giH.g"Go- Next, (b) is 
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proved as follows. Suppose Tig'Qo = Ti.g"Qo for g',g" G Q'. Then g' = hg"go for some 
h ETi and go E Qq, and thus we have g'Qo = hg"QQ. Now, since V intersects the orbit 
containing g"QQ at most once, we obtain g'Q^ = g"Qo- Therefore, we get g' = g" by 
our assumption. 

Sufficiency: Suppose Q = \_\gi(zgi'Hg'Qo. We want to show (a) V = {g'Go '■ 9' ^ G'} 
intersects each orbit under (H,Q/Qo) at least once; (b) V intersects each orbit at most 
once; and (c) g'Qo = g"Qo for g',g" G Q' implies g' = g". We begin by showing (a). 
Let gQo G Q/Qo, g ^ Q-, be arbitrarily given. Pick any gi G gQo C G = Ug'eg' ^fi''^o- 
Then gi can be written as gi = hg'gQ for some h & Ti, g' ^ Q' and g^ G Qq. Hence 
qGq = 9iGo = hg'Qo. This shows that the orbit containing gQo intersects V at least 
once. Next we verify (b). Suppose g'Qo = hg"QQ for g',g" G Q' and h E H. Then 
'Hg'Go = 'Hg"Qo, which implies g' = g" since Q = Ug'Ge' '^d'^o is a disjoint union. 
Hence we have g'Qo = g"Go. This observation shows (b). Finally, (c) can be verified 
similarly as (b). | 

Now, concerning the existence of V, we state the following theorem. 

Theorem 3.1. Suppose that there exists a global cross section Z for the action of Q 
on X, with the common isotropy subgroup denoted by Qq. Let Q' be a complete set of 
representatives of the double cosets TigQo, g ^ Q, in Q. Then a global cross section V 
exists for the action of Ti on Q/Qo if and only if 

nng'Qog'-', g'eg', 

are all conjugate to one another in Ti. 

Proof. First note that a global cross section V exists for action {Ti, Q / Qq) if and 
only if the isotropy subgroups 'higg^, g & Q, are all conjugate in H. 

Every g E Q = Ug'gg' '^d'Go can be written in the form g = hg'go, h E H, g' E 
G', go G Qo, and thus we have HggQ = Ti-hg'gogo — hHg'ggh~^. Therefore, V exists 
if and only if Hg/gg, g' G Q', are all conjugate in H. Here we can write Hg'gg as 

Hg^g, = {hen: hg'Go = g'Qo] = {hen: g'~'hg' eGo} = nn gVog'-'. I 

Remark 3.1. The condition that all Ti fl g'Qog'^^-, g' G Q', be conjugate does not 
depend on the choice of a complete set Q' . 

Let us now move on to the second problem — expressing V in a concrete form. The 
following theorem gives a useful explicit expression of V. We omit the proof because it 
is a simple consequence of Lemma 13.21 

Theorem 3.2. Suppose that there exists a global cross section Z for the action of Q 
on X, with the common isotropy subgroup Qo- Suppose furthermore that there exists a 
complete set Q' = {gi : i E 1} d Q of representatives of the double cosets HgQo, 9 ^ G, 
in Q such that Tig.gQ does not depend on i E I. Then 

V = {g^Go 

is a global cross section for the action of Ti on Q/Qo- 
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Remark 3.2. When the action of Ti on Q/Qo is free, any complete set of representa- 
tives of HgOo, g E G, in Q satisfies the condition of Q' in Theorem \3.l^ '^giGo — {^} 
for all i E I. 

Under the assumption of Theorem 13.21 X is decomposed as We now prove that 
V X Z in (jH)) is a global cross section for action (H, X). 

Theorem 3.3. Suppose that there exists a global cross section Z for the action of 
Q on X, with the common isotropy subgroup Qq. Suppose moreover that there exists a 
complete set Q' = {gi : i E /} of representatives of the double cosets HgQo, g & G, in 
Q satisfying the condition of Theorem VJ.'A and let V = {giQo \ i E I}. Then V x Z is 
in one-to-one correspondence with 

(9) Z := g'Z = {giZ : iel, zEZ}, 

and Z is a global cross section for the action of Ti. on X. 

Proof. It is easy to see that the correspondence {giQo, z) giZ between V x Z and 
Q' Z is a bijection. We show below that Q' Z is a global cross section for action {Ti, X). 

Let X be an arbitrary element of X . Then x can be written as x = gz, g E Q, z E 
Z. Furthermore, this g can be written a.s g = hgigo for some h E Ti, i E I and 
go E Qq. Hence, x = hgig^z = hgiZ and so Hx = TigiZ 3 giZ. This implies that Q'Z 
intersects each orbit Tix, x E X, under (7i, X) at least once. 

Next we show that Q'Z intersects each Hx, x E X, at most once. Suppose that 
hgiZ E G'Z for h E Ti, i E I and z E Z. Then there exist i' E I and z' E Z such 
that hgiZ = gi'z'. Since Z is a. cross section for action {Q,X), we have z = z' and 
hgiQo = gi'Go, which implies TigiQo = Tigi'Qo and thus i = i' . Therefore, we obtain 
hgiZ = giZ. This observation shows that Q'Z intersects each Tix, x E X, at most once. 

It remains to be proved that the isotropy subgroups Tig^z at the points giZ E Q'Z 
are all common. But this is obvious because Tig^z = Tig^g^ does not depend on i E I by 
the assumption of Theorem 13.21 I 

We call a global cross section Z for action {Ti, X) of the form © a decomposable 
global cross section. Of course, a general global cross section for {Ti, X) is not necessarily 
decomposable. 

Let us delve into Theorems 13.21 and 13.31 in two specific cases. 

First, consider the case where the action of ^ on A* is free. In that case, we want 
to decompose Q by considering the action of on ^ : {h,g) t— > kg, h E Ti, g eQ. 

Corollary 3.1. The action of Ti on Q is free, and any complete set {gi : i E 1} of 

representatives of the right cosets Tig, g E Q, in Q is a cross section for this action. 

Proof. It is trivial to see that action {Ti, Q) is free. The rest is obvious from Theorem 
Owith ^0 = {e}. I 

The latter statement of the corollary is also apparent from the fact that Tig, g E Q, 
are the orbits under {Ti,Q) and Q = 
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In the case of Corollary 13.11 Q is decomposed as 

g ^ nx{gr.tel} 

where T-t\Q is the right coset space 

n\g := {ng:geg} = {Hg, : z G /}. 

Corollary 3.2. Suppose that the action of g on X is free, and let Z he a cross 
section for this action. Then the action of H on X is free, and for any complete set 
G' = {9i '■ i ^ 1} of representatives of the right cosets Tig, g E g, in g, the set 
Z = g'Z = {giZ : i & I, z E Z} is a cross section for this action. 

Proof. It is evident that the action of ?^ on A" is free. The rest follows immediately 
from Theorem 13 . 31 with Lemma f3. II and Remark 13.21 | 

As an example, consider the actions related to the star-shaped distributions — the 
actions of ^ = M* (the multiplicative group of nonzero real numbers) and Ti = M!j_ (the 
multiplicative group of positive real numbers) on X = MP — {0} by scalar multiplication. 

In that case, g acts on X freely. Moreover, {±1} is a complete set of representatives 
of Hg, g E g, and is thus a cross section for action (H, g) by Corollarv l3.11 Accordingly, 
we have a one-to-one correspondence 

m; ^ m; X {±1}. 

Furthermore, we have by Corollarv 13.21 that Z = g'Z = Z U {—Z) with g' = {±1} 
is a cross section for the action oi Ti = on X = W — {0}. Let us take Z as 

Z = |(xi,...,a;p)' G§P-^ :xp>0}u|(a;i,...,a;p_i,0)' E : (xi, . . . , Xp-i)* G , 

where denotes the {p — l)-dimensional unit sphere and Z_i is a cross section for 

the action of ^ = M; on - {0}, G W-^. Since it is clear that g'Z = is 

true for p = 2, we see by induction on p that for all p, 

z = g'Z 

= |(xi,...,a;p)' G§P-i :xp^0}u{(a;i,...,xp_i,0)* G§P-i} 
= 

which is clearly a cross section for the action oi Ti = on X = W — {0}. In fact, we 
can take any Z (Z X = W — {0} which intersects each line through the origin in exactly 
one point. 

Next we treat the case which covers the two-sample Wishart problem. 

Corollary 3.3. Let g^ be a subgroup of g. Suppose that there exists a subgroup K, 
of g satisfying the following conditions: 
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(i) Every g E Q can be written uniquely in the form g = hk, /i G 7Y, k & K,. 

(ii) Qq is a subgroup of K. 

Then, the action of Ti on Q/Qq is free, and 

V = JC/Go = {kGo :ke}C} 

is a cross section for this action. 

Proof. Noting that Tifl/C = {e} by assumption (i) and that kQok~^ C /C, /c G /C, by 
assumption (ii), we have HnkQok~^ = {e} for any k E IC. Therefore, Ti^go — '^^d^og^^ 
is trivial for any in /C and thus for any g m Q : 

'HgQo = HhkGo = hHkgoh~^ = h{e}h''^ = {e}, g = hk, h eH, k e JC. 

Hence, action {TL,Q/Qq) is free. 

Let Q' d K be a complete set of representatives of the left cosets kQf), k E IC, in 
/C : /C = \_\gi^gi g'Qo- Then JC/Qo = {g'Qo '■ g' e Q'}, and by Theorem 13.21 it suffices to 
show that Q' is a complete set of representatives of the double cosets HgQo, g & G, in 
■ = Ug'eg' '^aVo- Since IJg'eS' T^dVo = ^dJ^'ee' dVo) = HK, = G, it remains to 
show that 

(10) ng'go = ng"go, g',g"eg', 

implies g' = g" . Suppose that (fTUj) holds. Then, there exist h E H and go G Qo such 
that g' = hg"gQ. By assumptions (i) and (ii), we have g' = g"go and thus g'Qo = g"Qo. 
The definition of Q' implies g' = g". I 

Remark 3.3. As was seen in the proof of Corollaru \3.'A if Q' d K, is a complete set 
of representatives of the left cosets kQo, k E JC, in /C, then Q' is also a complete set of 
representatives of the double cosets 'HgGoi g ^ Q, in Q. As can be shown in a similar 
manner, if we assume Q' C /C, the converse is true as well and the two conditions are in 
fact equivalent. 

In the case of Corollary EISl Q/Go is decomposed as 

<3/Go ^nx IC/Go. 

Corollary 3.4. Suppose that there exists a global cross section Z for the action of Q 
on X, with the common isotropy subgroup denoted by Qq. Suppose moreover that there 
exists a subgroup K, satisfying conditions (i) and (ii) in Corollaru VJ.'^ Then, the action 
of Ti on X is free, and Z = JCZ is a cross section for this action. 

Proof. Bv Lemma 13.11 and Corollarv 13.31 action is free. We show below that 

ICZ is a cross section for (7i, X). 

Let Q' C }C be some complete set of representatives of the left cosets kQo, k E )C, 
in /C. Then by Remark 13.31 Q' is a complete set of representatives of the double cosets 
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T^gGoi 9 ^ Qi in ^ as well. Theorem 13.31 implies that Z = Q'Z is a cross section for 
(7i, Af). Thus, the proof will be finished if we verify that Q'Z = K,Z. But this follows 
from fC = Ug'eg' 9' Go = QVo- I 

As an example, consider the situation related to the two-sample Wishart problem. 
Let Q = GL{p) (the general linear group) and 

(11) X = {{Wu W2) e PD{p) X PD{p) : 

the p roots of det(iyi - \{Wi + W2)) = are all distinct}. 

The action is (5, {WuW2)) ^ {BWiB\ BW2B'), BeGL{p). Let us take 

(12) Z = {(A, Jp - A) : A = diag(Ai, . . . , Ap), 1 > Ai > ■ ■ • > Ap > 0} , 
where Ip denotes the p x p identity matrix. Then we have 

(13) Go = {diag(ei, . . . , ep) : ei = ±1, . . . , Cp = ±1} . 

Now, as a subgroup of Q consider H = LT{p), the group of p x p lower triangular 
matrices with positive diagonal elements. Then the orthogonal group 0{p) can serve as 
the /C in Corollarv 13.31 With this /C, 

v = }c/go = {cgo:Ceo{p)} 

is the set of p x p orthogonal matrices with the sign of each column ignored. 

Corollarv 13.41 implies that the action oi Ti = LT{p) on X is free and that Z = K,Z 
with IC = 0{p) is a cross section for {Ti.,X). We can write Z as 

Z = ICZ 

= |(CAC', Ip-CAC') : C eO{p), A = diag(Ai,...,Ap), 1 > Ai > ■ ■ ■ > Ap > o} 
= {{U, Ip — U) : O < U < Ip, and the eigenvalues of U are all distinct} , 

where O denotes the null matrix and A < B means that B — A is positive definite for 
symmetric matrices A and B. 

3.2 Extended decomposable distribution 

In this subsection, we discuss the distributional aspect of the hierarchical decompositions 
in the preceding subsection. 

Suppose that there exists a subgroup £ of ^ of the form 

(14) c = g'Qo, 

where G' = {gi : i & 1} is a complete set of representatives of the double cosets 
'HgGo, 9 ^ G, in G such that Ti-g^gg does not depend on i E I. 
Then we have 

G = HG'Go = nc. 
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Therefore, every g & Q can be written in the form g = hi {or g = hl^^), h ETi, I E C 
Moreover, by considering the transitive action {{h,l),g) hgl~^, h E H, I E C, g E Q, 
of the product group H x C on Q, we have a bijection 

where = {[g , g) : g E T = C?^ is the isotropy subgroup at e eQ. 

Before proceeding further, let us see the two specific cases considered in the preceding 
subsection. 

First, consider the situation in Corollaries IH.ll and IH.2I Suppose that the action of Q 
on X is free and that a complete set Q' of representatives of the right cosets l-ig, g E Q, 
forms a subgroup of Q. Then Q' can serve as C. For instance, consider the example 
immediately after Corollary 13.21 — the actions related to the star-shaped distributions. 
Then Q' = {±1} forms a subgroup of ^ = M* and thus can serve as C. 

Next, consider the situation in Corollaries 13.31 and 13.41 Suppose that a subgroup /C 
of Q satisfies conditions (i) and (ii) of CoroUarv 13.31 and let Q' C K. be a complete set 
of representatives of the left cosets kQo, k E IC, in /C : /C = Ug'ee' d'^o- Then we have 
/C = Q'Qo, and Q' is a complete set of representatives of the double cosets HgQo, g E Q, 
in Q as well fRemark l3.3p . Thus /C can serve as C For instance, consider the example 
immediately after Corollarv 13.41 — the actions related to the two-sample Wishart problem. 
Then 0{p) can serve as C. 

Now we have by Theorem 13.21 and ()14|) that 

V = {g^Go ■.iEl] = {IGq -AeC] 

is a global cross section for action (7i, Q /Qq), and thus we obtain the decomposition 

X ^ U xV X Z, U = H/Ho, 
(15) X ^ {u,v,z), X = hlz, u = hHo, v = IQq, 

where TCq = HnQo, since £ contains e E Q and thus Qq E V. When Q' is taken in 
such a way that e E Q', we have Qo < Q'Qo = C and thus V = C/Qo is the left coset 
space. We can always take Q' in this way, and we decide to do so. 

Concerning topological questions, we make the following assumptions in addition to 
(a) through (d) at the beginning of Section 12.21 

Assumption 3.1. 

1. Ti and C are closed subgroups of Q. 

2. T is compact. 

Note that under our assumptions, TCq = TC H Qq is compact, since Qq is compact 
and H n Qo is closed in the relative topology of Qq. Note also that the one-to-one 
correspondence Q ^ {H x C)jT* is a homeomorphism since Q is second countable 
(p.92 of Wijsman (1990)). 

As before, let A be a relatively invariant measure on X under the action of Q with 
multiplier Now we define the extended decomposable distributions as follows. 
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Definition 3.1. A distribution on X is said to be an extended decomposable distribu- 
tion with respect to a pair of global cross sections {Z, V) if it is of the form 

f{x)X{dx) = fuiu{x))fviv{x))fz{z{x))X{dx). 

The following theorem gives the distributions of u, v and z when x is distributed 
according to an extended decomposable distribution. 

Theorem 3.4. Suppose that x is distributed according to an extended decomposable 
distribution fi({u{x)) f\;{v{x)) fz{z{x)) X{dx) . Then u = u{x) = hTCo, v = v{x) = IQo 
and z = z{x) (x = hlz) are independently distributed with the joint distribution 

fuiu)xiu)iJ,uidu) 

xfy{v)xiv)A^iv)A^iv)-'Mdv) 
xfziz)uz{dz), 

where A^ (resp. A'-') is the right-hand modulus of Q (resp. C), measure fxu (resp. 
Iiy) is a version of the invariant measures on lA = 'H/'Hq (resp. V = C/Qo), and vz is 
the probability measure in (jSj). 

This theorem can be proved by Proposition 7.6.1 and (7.6.5) of Wijsman (1990). 

4 Examples 

In this section, we apply Theorem 13.41 to the star-shaped distributions, the two-sample 
Wishart problem and the distributions of rankings. 

4.1 Star-shaped distributions with symmetry 

Consider the actions related to the star-shaped distributions — the actions of ^ = M* and 
n = RX on X = W - {0}. 

Then we have Qo = Hq = {1}. If we take C = Q' = {±1}, we obtain the bijection 

X ^ HxCxZ, 

X ^ {h, e, z), X = ehz, 

where ^ is a cross section for action {Q,X). Furthermore, Q = M* and C = {±1} 
are unimodular: = 1, A''' = 1. 

First, suppose that x is distributed according to a star-shaped distribution with 
respect to decomposable cross section Z = Q' Z = CZ = {ez : e = ±1, z E Z} = 
ZU{-Z) : 

f{hix))dx. 

This distribution can be regarded as the extended decomposable distribution with 

fu{h) = fn{h) = f{h), 
/v(e) = fc{e)^l, 
fz{z) ^ 1, 
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and 

X{dx) = dx. 

Dominating measure A is relatively invariant under the action of Q with multiplier 

x{9) = \9\'', geg, 

where | • | denotes the absolute value. Therefore, we have by Theorem 13.41 that /i, e and 
z are independently distributed according to 

(16) -f{h)hPh~^dh = -f{h)hP-^dh, fiv = lie and 

Co Co 

respectively, where cq = f{h)h^~^dh and fic{{^}) = ^J'c{{—^}) = 1/2. We can see 
that the distributions of x and —x are the same by —x^{h,—e,z) for x<-*{h,e,z). 
Under the additional assumption that h{x) is piecewise of class C^, we have 

vz{dz) = 2co{z,nz)dz, 

where nz is the outward unit normal vector of Z, dz on the right-hand side is the 
volume element of Z and ( ■ , ■ ) denotes the standard inner product (Section 4 of 
Kamiya, Takemura and Kuriki (2006)). 

Next, by taking nonconstant /^(e) instead, we can make an asymmetric distribution 
of X as follows. Suppose 

(17) X ~ c{e{x))f{h{x))dx, 
where 

(18) c(6) := 

with < c < 2. Then again Theorem 13.41 implies that h, e and z are independently 
distributed as (fTH|) . but this time with /iy = fic replaced by /i£({l}) = c/2, flc{{—l}) = 
1 — (c/2). In this case, the distribution of —x is different from that of x unless c = 1. 

Note that we can make a more general skewed distribution by considering a cross- 
sectionally contoured distribution f{e{x)h{x))dx with respect to Z. But in that case, 
we have to specify the function / defined on the whole of Rx = (— oo, 0) U (0, oo). By 
contrast, in the case of (jl7|) we have only to specify / defined on = (0, oo) and one 
value c in (fTH|l. 

4.2 Two-sample Wishart problem 

Consider the action of ^ = GL{p) on the sample space X in (lllj). The cross section Z 
is taken as (fT^ with the isotropy subgroup Qo in (fT^. 

We continue to take H = LT{p) and C = IC = 0{p). Then we obtain the bijection 

X ^ Hx C/Go X Z, 
{W,,W2) ^ (T, CQo, (A, /p-A)), {W,,W2)= (tCAC't\ TC{Ip-K)C'ty 
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Furthermore, Q = GL{p) and C = 0{p) are unimodular: = 1, = 1. 

Suppose that the random matrices Wi and W2 are independently distributed ac- 
cording to Wp{ni,T,) and Wp{n2,T,), respectively. Then the distribution of (PFi,VF2) 
can be regarded as the extended decomposable distribution with 

fuiT) = fn{T) oc etr (^-h^-^TT'^ , fv{Cg,) = fc/GoiCQo) = 1, fz ((A, Ip - A)) = 1 
and 

(19) X{d{Wi,W2)) = {detWiy-'^{detW2)''~'^dWidW2, 

where a = ni/2, b = ^2/2, Wi = W2 = (w2,ii), dWi = Hi^jdwi^ij, dW2 = 

Y[i>j d'W2,ij- The dominating measure X{d(Wi,W2)) is relatively invariant with multiplier 

X{B) = (det = (det 5)"!+"% B G GL{p), 

(Wijsman (1990), (9.1.4)). It follows from Theorem 13.41 that T, CQo and A are inde- 
pendently distributed. The distributions of these parts are given in standard textbooks 
of multivariate statistical theory (see Anderson (2003) or Muirhead (1982), for exam- 
ple). In particular, CQo is distributed according to the invariant probability measure on 
V = C/Qo = 0{p)/Qo induced by the Haar measure on 0{p). 

A nonstandard distribution is given as follows. Since the normalizer of Qo is 

A/" = {P G GL{p) : P has exactly one nonzero element 

in each row and in each column}, 

we know from Q that a general global cross section Z' is of the form 

(20) Z' =1 (^fiP(A)AP(A)'5*, BP{A){Ip - A)P(A)*fi*) : 

A = diag(Ai, . . . , Ap), 1 > Ai > • ■ ■ > Ap > o| 

with B G GL{p) and P(A) G TV. Without loss of generality, we assume B = Ip in (plj) . 
Let B(W) denote the equivariant part of W = {Wi,W2) with respect to Z. Then the 
equivariant part with respect to Z' is B(W)P{A(W))^^Qo by (jl}. Writing the latter as 

B{W)P{A{W)r'go = T'{W)G'{W)go, T'{W) G LT{p), G'{W) G 0{p), 

we obtain the decomposition 

W^iT'iW), C'iW)go, z'iW)), 

where z'{W) = {P{A{W))A{W)P{A{W))\ P{A{W)){Ip- A{W))P{A{W)y) G Z' is the 
invariant part of W with respect to Z'. Suppose that the density f(W) with respect 
to A in (Uni) with general a,b> {p + l)/2 is factored as 

f{W) = fn{T'{W))fc/g,{C'{W)go)fz'{z'{W)). 
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Then we can get from Theorem l3.4l the distributions of the three parts T' = T'{W), C'Qq = 
C'{W)go and z' = z'{W) using = = 1, xiB) = (det5)2("+^) and finidT) = 
f^LTip)idT) = Uti^^'dT, dT = Ui>jdU„ T = (t,,) (Wijsman (1990), (7.7.2)). Specifi- 
cally, 

^« 7=1 

C'Go ~ fc/go{C'Qo)dfic/go{C'Qo), 
z' -fz'{z')duz'{z'), 

CZ' 

where ch = Jlt{p) f'HiT)]Yi^itii^'^^''dT, cz' = J^, fz'iz)diyz'iz) and fic/Go is an ap- 
propriate version of the invariant measures on C/Qq = 0{p)/Qo. 

4.3 Decompositions of rankings 

Our general discussion can be applied to discrete distributions as well. In this subsection, 
let us look at one such example — distributions of preference rankings. For the analysis 
of ranking data in general, the reader is referred to the excellent books by Critchlow 
(1985), Diaconis (1988) and Marden (1995). Other interesting problems about preference 
rankings can be found in Kamiya, Orlik, Takemura and Terao (2006). 

Let us consider rankings of m objects l,2,...,m. We denote rankings as a = 
((t(1), o"(2), . . . , cr(m)), where a{i) stands for the rank given to object i. Then we can 
regard 

a = (a(l),a(2),...,a(m))=^^ a(^2) a^) 

as an element of the symmetric group 5*^ on {1,2,..., m}. We can deal with distribu- 
tions of rankings a G Sm by considering probability functions on X = Sm- 

Here we define an action of ^ = 5*^-1 < 5*^ on X = Sm as follows. Thinking of 
T G Sm-1 as permutations of {2, . . . , m}, we express r as 

r = (r(2).....r(™))=(^22, Z . 

Permutation r G Sm-i changes rank A; G {2, . . . ,m} to rank T{k) G {2, . . . ,m}. Now 
we consider the action (r, a) i-^ ra. Here ra G Sm means {Ta){i) = T{a{i)), i G 
{1, 2, . . . , m}, where we agree that r(l) = 1. Note that this action is free: Qo = {e}, e = 
(2,...,m) = (l,2,...,m). 

Under the above action, the orbit containing a G Sm is 

Qa = Sm-io- = {a e Sm - 5-^-^(1) = cr'\l)} 

and X = Sm consists of the m orbits 

{a e Sm ■ dr{i) = 1}, i = 1,2, . . . ,m. 
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Orbit {a G Sm '■ d'{i) = 1} is seen to be the set of all rankings that rank object i first, 
and we will call this the z-orbit. 

One way of selecting a representative (jj of the i-orbit is choosing (jj as 

(21) ^;\l)=^, ar\2)<---<ai\m). 

Let us take the cross section Z consisting of these representatives (Tj, i = 1, 2, . . . , m, 
i.e., Z = {ai -.1 = 1,2,..., m}. Then, with respect to this cross section, we can write an 
arbitrary a E X = Sm uniquely as 

a = TS = r(cr)s(cr), T E Q = Sm-i, S E Z. 

Note that s{a) can be expressed explicitly as s{cr) = a„-i(^iy For example, when m = 4, 
ranking 




belongs to the 3-orbit, and the corresponding r(cr) and s(cr) are given as 

r(cr) = (4,2,3)= (2 3 4^^ ,(a) = ^3 = (2, 3, 1, 4) = ^ 3 4^ _ 

We are now in a position to introduce a family of distributions on X = Sm- Consider 
the following type of probability functions: 

(22) p{cr) = pg{d{a, s{cr)))pzis{a)), a G Sm, 

where d{ ■ , ■ ) is a right-invariant metric on Sm-i (note that cr^^{l) = s{a)^^{l)). 
We usually take pg to be a decreasing function. In that case, pg{d{a, s{a))) expresses 
a unimodal distribution in the orbit with the representative s{a) as the modal ranking; 
the farther away from the representative s{a), the smaller probability of that ranking 
cr. An example of a specification of pg is provided by Mallows' model pg{d{a, s{a))) = 
cexp{6d{a, s{a))) with < 0, where c is the normalizing constant (Mallows (1957), Fei- 
gin and Cohen (1978), Fligner and Verducci (1986)). Now, writing pg(r) := pg{d{T,e)), 
we can express (j22I) as 

= PgiTi(^))pzis{a)), a G Sm- 

This can be regarded as a decomposable distribution under our action of ^ = 5^-1 on 
X = Sm- We are writing p{a) instead of f{a) in order to emphasize that we are dealing 
with discrete probability functions. Of course, the density is with respect to the counting 
measure, so x = 1- 

In the discussions so far, the representatives ai, . . . ,am of the orbits have been chosen 
as (jUj), but this is just one way of choosing them and other choices are also possible. The 
choice is arbitrary as long as they form a complete set of representatives. In fact, the 
selection rule can differ from orbit to orbit. For example, we may consider taking the 
representative a^^ G {cr G Sm '- cr(zo) = 1} for some object io G {1, 2, . . . , m} as follows: 
for some object jo, 

(23) <(l)=^o, <(2)<---<a-i(m-l), a-\m) = jo- 
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For the other orbits, we continue to select representatives CTj, % ^ as in (j^U- 

We now discuss motivations of the above modehng. Imagine we are considering peo- 
ple's preference rankings of a league of m sports teams l,2,...,m. Suppose that we 
are faced with the following situation: while people are interested in their favorite team, 
they do not care much about the differences among the rest and tend to simply rank the 
second to last preferred teams according to the ranks in the standings (based on winning 
percentages). In that case, we can describe the situation by modeling the distribution of 
the rankings as follows. 

We label the m teams with the ranks in the standings. Then, we choose the repre- 
sentatives of all the orbits as in (|2H) . With these choices, the distribution (j22j) of people's 
preference rankings implies that their top rank is distributed according to ^^(^(cr)), while 
the rest of the ranks are distributed based on the distances to the modal ranking, which 
in this case is the ranking (having the same relative ranks of the non-top objects as the 
ranking) in the standings; the closer to the ranking in the standings, the larger percentage 
of people with that ranking. So we have succeeded in describing the case in question. 

Alternatively, there may be some cases where the fans of a certain team %q have a 
strong sense of rivalry with some other team jo- In those cases, the choice fj23|) will be 
more appropriate. 

Now, we can go further and consider decompositions into three parts. Suppose 
that people are very interested in the top rank, interested to some degree in ranks 
2, . . . , m' (2 < vn! < m — 1) and totally indifferent about the rest of the ranks m' -|- 
1, . . . ,m. 

Let us consider the action of TC = Sm-m' on Q = Sm-i, with Sm-m' being regarded 
as the set of permutations of {m' -|- 1, . . . , m}. Denote a cross section of this action by 
V. Then any t E Q can be written uniquely as t = ht, h & TC, t E V. Hereafter we 
will write S.^ = Sji,. ..,„}, 

Sm-l — S^2,...,m} and Sm-m' — S^m'+l,...,m}- 

By writing a & X = S{i^,,,^m} as 

a = Ts (t eQ = 5'{2,. ..,„}, s e Z) 

= htS {h eH = S{m'+l,...,m}, t G V), 

we obtain bijections 

a ^ (r, S{2,...,m}0') (e S{2,...,m} X 5'{2,...,m}\5'{i,...,m}) 

^ {h, 5'{m'+l,...,m}T, S{2,...,m}0') 

(24) (G S{m'+l,...,m} X 'S'{.m'+l,...,m}\5'{2,...,m} X S{2,...,m}\S{l,...,m}) 

<-> {h, t, S) e S{m'+l,...,m} X V X Z. 

Remember that 7Y\^ for Ti. < Q denotes the right coset space: T-C\Q = {Tig : g E Q}. 

Example 4.1. Let m = 6, m' = 3, and take Z = {cTj : i = 1, 2, . . . , 6} with ai, . . . , ctq 
as in (PT|) and V = G S'{2 6} • {hj} C {2,..., 6}, i ^ j} with <J(ij) such that 

= = j'^ % •)(4) < ^(7i)(5) < a^^^{6). Then, 

A 2345 6\ _ /2 345 6\/l2345 6\ 
V6 35 142y) ~ ^6 3542^^2 341 56^ 

A 5 6\ /2 3 4 5 6\ A 2 3 4 5 6\ 
^6 54^)1^4 3562^)1^2 341 56^' 
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so 

„ A 2 3 4 5 6\ 

^2,3,4,5,6} 3 ^6 3 5 1 4 2j 

//2 3 4 5 6\ „ A 2 3 4 5 6\\ 

^1^6 3 5 4 2j' ^{2.3,4,5,6} 3 5 1 4 2JJ 

'S'{2,3,4,5,6} X 'S'{2,3,4,5,6}\'S'{1, 2,3,4,5,6}) 

f fA 5 6\ ^ /2345 6\ „ A2345 6\\ 

^6 5 4J' ^{4'5'6}^6 3 5 4 2J' ^{2,3,4,5,6} 3 5 1 4 2JJ 

'S'{4_5^6} 'S'{4^5^6}\'S'{2,3,4,5,6} ^ 'S'{2,3,4,5,6}\'S'{1, 2,3,4,5,6}) 

/A 5 6\ A 3 4 5 6\ A 2 3 4 5 Q\\ ^ ^ 
^1^6 5 4J' ^4 3 5 6 2J' ^2 3 4 1 5 q) ) ^ x ^ x 

Now consider probability functions p{a) which can be factored with respect to (pijl 

as 

(25) p{a) = Pn\giS{,ri'+i,...,m}'^)Pzis) 

with Pn\g{S{m'+i,...,ni}'^) ■= Pn\gid'iS{m'+i,...,m}'^, S{m'+i,...,m})), where d' is the Haus- 
dorff metric on H\Q = S{rn'+i,...,m}\S{2,...,m} induced by the metric d on Q = 3^2,. ..,m} 
(Critchlow (1985)). Note that distribution can be seen as an extended decomposable 
distribution with respect to (j24j) . Now, label the teams with the ranks in the standings 
and take Z = {o", : z = 1, 2, . . . , m} with (jj, z = 1, 2, . . . , m, in (PT|) as before. Then 
(I25|l describes the situation stated earlier. Note that in the choice of V is irrelevant, 
because the rest of the ranks are uniform. 

Remark 4.1. Here we have studied the permutations of ranks. By considering the order- 
ings 

(top ranked object, second ranked object, . . . ) 

instead of rankings 

[object I's rank, object 2's rank, . . . ), 
we can also deal with the permutations of objects in a similar manner. 
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